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The Foundation began with the aspirational goal that all children should grow up in economically successful families, live in 


supportive communities and meet developmental, health and educational milestones. In an effort to capture the complex set 
of factors that influence a child’s success, we chose to develop a composite index that would allow comparisons across racial 
and ethnic groups at the national level and within and across states. 


‘The selection of indicators in the Race for Results Index was heavily informed by research from the Brookings Institution's 
Social Genome Project, which connects key indicators to the likelihood of a young person becoming middle class by middle 
age, and by the research that shows that children do best in supportive families and communities. The 12 indicators selected 
reflect different dimensions of child well-being and cover different stages of development, from birth to young adulthood. 

The indicators establish a positive frame, focusing on aspirational goals and the key pathways to success for children and youth 
in different age groups. 


THE INDEX: SECOND EDITION 2017 RACE FOR RESULTS INDEX METHODOLOGY 


We have developed the 2017 edition of the index with one noteworthy change. 


The way that the high school students graduating on time indicator is calculated and measured for the index was changed. 

In the 2014 report, we derived this indicator from the Averaged Freshman Graduation Rate (AFGR). For the 2017 report, 

we measure this indicator using data derived from the Adjusted Cohort Graduation Rate (ACGR). We changed the data source 
because the ACGR has now become the “gold standard” for graduation-rate reporting. It was designed to be more accurate 
and uniform across states, and it is the number that the U.S. Department of Education uses to report graduation rates. 


THE FOLLOWING 12 MEASURES MAKE UP THE INDEX: 


¢ Babies born at normal birthweight e Young adults ages 19 to 26 who are in school or working 
e Children ages 3 to 5 enrolled in nursery school, ¢ Young adults ages 25 to 29 who have completed an 
preschool or kindergarten associate’s degree or higher 
e Fourth graders who scored at or above proficient ¢ — Children birth to 17 who live with a householder 
in reading who has at least a high school diploma 
e Eighth graders who scored at or above proficient e Children birth to 17 who live in two-parent families 
in math ¢ — Children birth to 17 who live in families with incomes 
¢ High school students graduating on time at or above 200 percent of poverty 
e Females ages 15 to 19 who delay childbearing e Children birth to 17 who live in low-poverty areas 
until adulthood (poverty <20 percent) 


In selecting indicators to include in the index, one of the most important considerations was the availability of data for various 
racial and ethnic groups. For purposes of this analysis, all racial and ethnic groups are mutually exclusive. All data for racial 


groups are reported for non-Hispanics only, based on the U.S. Office of Management and Budget’s guidelines:! 


e ~~ African American e Latino 
e American Indian ¢ White 
e Asian and Pacific Islander ¢ ‘Two or More Races 


Due to the relatively small size of the population and constraints on several of the data sets, Asian and Pacific Islander children 
were combined into one group. There are no state-level indices for children and youth of two or more races because insufficient 
data were available to allow meaningful comparisons on all 12 measures (missing data for babies born at normal birthweight, 
high school students graduating on time and females ages 15 to 19 who delay childbearing until adulthood). However, we 
present national estimates for children and youth of two or more races for the nine individual indicators with available data. 


DEVELOPING A RACIAL AND ETHNIC INDEX OF WELL-BEING 


2017 RACE FOR RESULTS INDEX METHODOLOGY 


Developing a Racial and Ethnic Index of Well-Being 


There was considerable discussion about the appropriate 
method that should be used to construct the racial and 
ethnic index. Staff from the Annie E. Casey Foundation 

and Population Reference Bureau (PRB) did an extensive 
literature review, held several internal meetings and consulted 
with expert advisors before selecting the final methodology. 
We initially considered constructing index values by 
comparing the well-being of children in different racial 

and ethnic groups against the national average. However, 
the national average does not provide a very compelling 
comparison point or benchmark for measuring success. 

We also considered constructing a simple average of the 

12 indicators of well-being. But this method proved to be 
problematic because of the wide-ranging values across the 
different measures. In the end, we chose to replicate the 
method used to construct the annual KIDS COUNT Index, 
which is based on standardized scores. 


‘The KIDS COUNT Data Book includes an index derived 
from standard scores (or z-scores) based on 50-state averages 
and standard deviations for 16 indicators of well-being. PRB 
staff created a modified version of the KIDS COUNT Index 
by calculating index scores for each racial and ethnic group 

in each state. For each variable, standard scores were derived 
by subtracting the mean state value (across 50 states and five 
racial and ethnic groups) from the observed estimate for a 
given state and racial and ethnic group and dividing the result 
by the standard deviation for that distribution of estimates, as 
shown in the following formula: 


In this formula, z represents the z-score for a given state 
(s) and racial and ethnic group (7), and x represents the 
estimate of child well-being for a given state (s) and racial 
and ethnic group (7). The Greek letter mu represents the 
mean across the 250 state values (50 states * 5 racial and 
ethnic groups), and the Greek letter sigma represents the 
standard deviation. 


Standardization is useful for this because although all the 
indicators are expressed as percentages the range of values 
across the 12 measures is very different. For example, the 
percentage of African-American babies born at normal 
birthweight is highest in North Dakota at 93 percent, just 
nine percentage points higher than the share in Alabama 

and Mississippi, which tied for the lowest rate at 84 percent. 
But the highest percentage of African-American eighth 
graders who scored at or above proficient in math (22 percent 
in Massachusetts) is more than four times the lowest rates 

(5 percent in Alabama and Michigan). By standardizing these 
variables and putting them on the same scale, we are able to 
capture this state-level variation and account for the fact that 
percentage values have different meanings across indicators. 


The main drawback of using standard scores is that the 
resulting index values are difficult to interpret. Therefore, to 
show the differences in values across racial and ethnic groups 
and states, we converted these z-scores to a scale ranging from 
zero to 1,000, using the following formula: 


f= ( Bsr — Zmin )* 1,000 


max — min 


In this formula, frepresents the final index value for a given 
indicator, state (s) and racial and ethnic group (7), while z 
represents the z-score calculated in the previous formula for 
a given state (s) and racial and ethnic group (7). Zin and 
Zax tepresent the smallest and largest z-scores, respectively, 
across all racial and ethnic groups and states for a given 
indicator. Index values are put on a scale ranging from zero 
to 1,000. We used a scale from zero to 1,000 (instead of 
zero to 100) so that people would not misinterpret the index 
values as percentages. Lower values represent worse outcomes 
for children, while higher values represent more positive 
outcomes for children. 


We then constructed national index values by weighting the 
state-level index values for each racial and ethnic group based 
on the share of the national population under 18 in that 
group in each state. 


MISSING DATA AND DATA SUPPRESSION 


Missing Data and Data Suppression 
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Missing data pose a potential problem because if data are missing for even one indicator for a given state and racial and ethnic 


group, then we are unable to construct complete summary index values for that state and group. This issue is most likely to 


affect states with smaller populations and fewer racial and ethnic groups. For example, it would not be possible to produce 


reliable estimates for Latino children in Maine for all 12 indicators of well-being. 


We present state-level index values for Latinos and the following non-Hispanic racial groups: African Americans, American 


Indians, Asians and Pacific Islanders and whites. However, state-level index values for these groups are not displayed if four or 


more of the 12 estimates in the index are deemed to be unreliable. We present national-level index values for these same racial 


and ethnic groups. National and state index values are not calculated for those who identified as two or more races because too 


many estimates are missing to construct reliable index values. 


Multiyear estimates were used where possible to boost the sample size and increase data reliability. Seven of the 12 indicators are 


based on three- and five-year estimates from the American Community Survey (six indicators were developed using three-year 


estimates and five-year estimates were used for children who live in low-poverty areas). 


EVALUATING SURVEY DATA 


Nine of the 12 indicators are based on survey data and 
therefore have some degree of error. 


The following is a list of those nine indicators and their 
respective data source: 


National Center for Education Statistics, National Assessment 
of Educational Progress (NAEP) 


e Fourth graders who scored at or above proficient 


in reading 


e Eighth graders who scored at or above proficient in math 


United States Census Bureau, American Community Survey (ACS) 


e Children ages 3 to 5 enrolled in nursery school, 
preschool or kindergarten 


e Young adults ages 19 to 26 who are in school or working 


¢ Young adults ages 25 to 29 who have completed an 
associate’s degree or higher 


e Children birth to 17 who live with a householder 
who has at least a high school diploma 


e Children birth to 17 who live in two-parent families 


e — Children birth to 17 who live in families with incomes 
at or above 200 percent of poverty 


e — Children birth to 17 who live in low-poverty areas 
(poverty <20 percent) 


Since all the indicators in this index are expressed as 
percentages, we use the coefficient of variation (CV) to 
determine whether an estimate should be suppressed. 
The CV is calculated using the following formula: 


_ SE,, 


CV. * 100 


sr 


In this formula, the coefficient of variation (CV) for each state 
(s) and racial and ethnic group (7) is calculated by dividing the 
standard error (SE) of the estimate by the estimate itself and 
multiplying the result by 100. An estimate with a small CV 
is considered more reliable when compared with an estimate 
with a larger CV. There are no strict rules for an exact cutoff 
point for a reliable CV. Generally, smaller CVs are better. The 
Centers for Disease Control and Prevention reviewed the 
criteria for data suppression used by 22 of the 23 major data 
systems (Healthy People 2010 Criteria for Data Suppression) 
and found that 30 percent is a common CV level used to 
suppress estimates. Thirty percent is the threshold we used 
in our analysis to suppress unreliable estimates. 


LIMITATIONS 


ADMINISTRATIVE DATA 


The remaining three indicators are collected from 


administrative data sources: 


National Center for Health Statistics, National Vital Statistics 
System — Final Birth Data 


e Babies born at normal birthweight 


e Females ages 15 to 19 who delay childbearing 
until adulthood 


National Center for Education Statistics, Common Core of Data (CCD) 


¢ — High school students graduating on time 


Limitations 
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Because these three measures are based on administrative 
data and theoretically comprise the entire population of 
their respective groups, results are not statistically evaluated 
to determine the reliability. However, in accordance with 
guidelines established by the National Center for Health 
Statistics, we suppress all data that are based on fewer than 
20 events. Also in accordance with those guidelines, we 
suppress results where the inverse of the number of events 
is fewer than 20. For example, in a given state where 205 of 
the total 220 babies born were of normal birthweight, the 
result would be suppressed since the remaining 15 were low 
birthweight and do not meet the threshold of 20. Results 
from the Common Core of Data are not subject to these 
guidelines, although all the results did meet these criteria. 


Comparisons should not be made to results from the 2014 Race for Results report. We do not recommend comparing 


the Race for Results Index scores over time for the following reasons: 


|. Changes in the indicators: Between the 2014 and 2017 


reports, as previously discussed, we changed the indicator 


that measures the percentage of high school students 
graduating on time. The Adjusted Cohort Graduation 
Rate (ACGR) now replaces the Averaged Freshman 
Graduation Rate (AFGR). Although the ACGR and the 
AFGR measure a similar outcome — the share of high 
school students who graduate within four years — the 
methods for calculating each measure are quite different. 
This means that the reported percentages of students 
graduating on time and states’ relative graduation-rate 
ranking may vary across the two measures. These types 
of differences can result in states having a different index 
score because of the methodology change rather than 
due to any meaningful changes in how children are 
faring within and across states. Accurate discussions of 
year-to-year variations in index scores must be the result 
of changes that can only be attributed to shifts in how 
children are faring. 


2. Measurement error: Data disaggregated by race and 
ethnicity will have more measurement error than data 
examining all children. Additionally, changes in racial 
and ethnic disparities tend to move slowly. To accurately 


attribute changes in index scores to changes in racial 
gaps, we need to rule out the possibility of measurement 
error. The best way to do this is to have a long-time span 
between comparison points. For most indicators in the 
index, the gap between data periods is three or fewer 
years. This is not a long enough time trend to help us 


rule out measurement error. 


3. Index construction/use of the total variance: There are many 
components to creating the index scores (results for 
12 indicators that are then transformed two different 
ways prior to being combined into a total index score). 
‘This means when an index score changes across time, it 
is difficult to understand what caused the change. For 
example, in the first transformation, the indicators are 
re-scaled using the total variance for that indicator for 
the United States. A year-to-year change in the total 
variance would lead to a change in index scores that 
may have nothing to do with meaningful changes in 
children’s well-being. We then use the gap between the 
best performing state and race combination and the 
worst performing state and race combination to re-scale 
the indicators on a scale of zero to 1,000 (where zero is 
doing poorly compared with other states, 500 is about 
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average and 1,000 is doing the best). Because this scale between the best and worst state or race combination 

is fixed, all scores will always be on a zero to 1,000 scale on a single indicator) that may not have much meaning 
and year-to-year numerical changes on this scale could for how the well-being for a particular racial or ethnic 
be due to myriad reasons (e.g., a change in the gap group changed. 


THERE ARE TWO COMPARISONS ACROSS TIME THAT CAN BE MADE. 


|. Within racial and ethnic groups across states: Comparisons to how a state ranked for a racial and ethnic group relative to other 
states can be made, as well as describing how that order has changed over time. Because of the nature of rankings, it is not 
usually possible to pinpoint why a state’s rank changed from one specific rank to another. 


2. _ Racial and ethnic disparities within states: Users can compare the gap between racial and ethnic groups within a state across time 
(e.g., comparing non-Hispanic blacks and non-Hispanic whites within a state). Users can note whether those gaps have 
changed markedly from the 2014 report to the 2017 report. No change to relatively small changes mean that things have 
stayed more or less the same, but a really large change signals that something shifted in that state across time. 


OTHER LIMITATIONS 


Because we needed to include data that were comparably collected in every state and included a large enough sample to provide 
valid estimates for the five largest racial groups, we were unable to incorporate some key components that impact a child’s 
successful transition into adulthood. For example, we were not able to find data that met our criteria to measure a young 
person’s involvement in the juvenile justice system — an indicator that disproportionately impacts children of color’s chances of 
success. In addition to limited data availability, the data included in the Race for Results Index are not longitudinal. Therefore, 
we are able to paint a picture of a point in time for groups of children, but cannot see how indicators build on each other from 
birth into middle age to actually track a child’s trajectory over time. Finally, we did our best to suppress estimates that were 
deemed unreliable, but there might be instances — especially in smaller states with fewer racial and ethnic groups — where 
published estimates are volatile. 


|. Office of Management and Budget. (1997). Revisions to the standards for the classification of federal data on race and ethnicity. Retrieved from 
www.whitehouse.gov/omb/fedreg_1997standards 
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